Conditional Partial Plans for Rational Situated Agents Capable of Deductive Reasoning and Inductive Learning

Rational, autonomous agents that are able to achieve their goals in dynamic, partially observable environments are the ultimate dream of Artificial Intelligence research since its beginning. The goal of this PhD thesis is to propose, develop and evaluate a framework well suited for creating intelligent agents that would be able to learn from experience, thus becoming more efficient at solving their tasks. We aim to create an agent able to function in adverse environments that it only partially understands. We are convinced that symbolic knowledge representations are the best way to achieve such versatility. In order to balance deliberation and acting, our agent needs to be emph{time-aware}, i.e. it needs to have the means to estimate its own reasoning and acting time. One of the crucial challenges is to ensure smooth interactions between the agent's internal reasoning mechanism and the learning system used to improve its behaviour. In order to address it, our agent will create several different conditional partial plans and reason about the potential usefulness of each one. Moreover it will generalise whatever experience it gathers and use it when solving subsequent, similar, problem instances. In this thesis we present on the conceptual level an architecture for rational agents, as well as implementation-based experimental results confirming that a successful lifelong learning of an autonomous artificial agent can be achieved using it

Corrupted Contextual Bandits with Action Order Constraints

We consider a variant of the novel contextual bandit problem with corrupted context, which we call the contextual bandit problem with corrupted context and action correlation, where actions exhibit a relationship structure that can be exploited to guide the exploration of viable next decisions. Our setting is primarily motivated by adaptive mobile health interventions and related applications, where users might transitions through different stages requiring more targeted action selection approaches. In such settings, keeping user engagement is paramount for the success of interventions and therefore it is vital to provide relevant recommendations in a timely manner. The context provided by users might not always be informative at every decision point and standard contextual approaches to action selection will incur high regret. We propose a meta-algorithm using a referee that dynamically combines the policies of a contextual bandit and multi-armed bandit, similar to previous work, as wells as a simple correlation mechanism that captures action to action transition probabilities allowing for more efficient exploration of time-correlated actions. We evaluate empirically the performance of said algorithm on a simulation where the sequence of best actions is determined by a hidden state that evolves in a Markovian manner. We show that the proposed meta-algorithm improves upon regret in situations where the performance of both policies varies such that one is strictly superior to the other for a given time period. To demonstrate that our setting has relevant practical applicability, we evaluate our method on several real world data sets, clearly showing better empirical performance compared to a set of simple algorithms

Wisdom of the Contexts: Active Ensemble Learning for Contextual Anomaly Detection

In contextual anomaly detection (CAD), an object is only considered anomalous within a specific context. Most existing methods for CAD use a single context based on a set of user-specified contextual features. However, identifying the right context can be very challenging in practice, especially in datasets, with a large number of attributes. Furthermore, in real-world systems, there might be multiple anomalies that occur in different contexts and, therefore, require a combination of several "useful" contexts to unveil them. In this work, we leverage active learning and ensembles to effectively detect complex contextual anomalies in situations where the true contextual and behavioral attributes are unknown. We propose a novel approach, called WisCon (Wisdom of the Contexts), that automatically creates contexts from the feature set. Our method constructs an ensemble of multiple contexts, with varying importance scores, based on the assumption that not all useful contexts are equally so. Experiments show that WisCon significantly outperforms existing baselines in different categories (i.e., active classifiers, unsupervised contextual and non-contextual anomaly detectors, and supervised classifiers) on seven datasets. Furthermore, the results support our initial hypothesis that there is no single perfect context that successfully uncovers all kinds of contextual anomalies, and leveraging the "wisdom" of multiple contexts is necessary.Comment: Submitted to IEEE TKD

Stream data cleaning for dynamic line rating application

The maximum current that an overhead transmission line can continuously carry depends on external weather conditions, most commonly obtained from real-time streaming weather sensors. The accuracy of the sensor data is very important in order to avoid problems such as overheating. Furthermore, faulty sensor readings may cause operators to limit or even stop the energy production from renewable sources in radial networks. This paper presents a method for detecting and replacing sequences of consecutive faulty data originating from streaming weather sensors. The method is based on a combination of (a) a set of constraints obtained from derivatives in consecutive data, and (b) association rules that are automatically generated from historical data. In smart grids, a large amount of historical data from different weather stations are available but rarely used. In this work, we show that mining and analyzing this historical data provides valuable information that can be used for detecting and replacing faulty sensor readings. We compare the result of the proposed method against the exponentially weighted moving average and vector autoregression models. Experiments on data sets with real and synthetic errors demonstrate the good performance of the proposed method for monitoring weather sensors.This research was partially funded by Spanish Government under Spanish R+D initiative with reference ENE2013-42720-R and RETOS RTC-2015-3795-3

Explainable Predictive Maintenance

Explainable Artificial Intelligence (XAI) fills the role of a critical interface fostering interactions between sophisticated intelligent systems and diverse individuals, including data scientists, domain experts, end-users, and more. It aids in deciphering the intricate internal mechanisms of ``black box'' Machine Learning (ML), rendering the reasons behind their decisions more understandable. However, current research in XAI primarily focuses on two aspects; ways to facilitate user trust, or to debug and refine the ML model. The majority of it falls short of recognising the diverse types of explanations needed in broader contexts, as different users and varied application areas necessitate solutions tailored to their specific needs. One such domain is Predictive Maintenance (PdM), an exploding area of research under the Industry 4.0 \& 5.0 umbrella. This position paper highlights the gap between existing XAI methodologies and the specific requirements for explanations within industrial applications, particularly the Predictive Maintenance field. Despite explainability's crucial role, this subject remains a relatively under-explored area, making this paper a pioneering attempt to bring relevant challenges to the research community's attention. We provide an overview of predictive maintenance tasks and accentuate the need and varying purposes for corresponding explanations. We then list and describe XAI techniques commonly employed in the literature, discussing their suitability for PdM tasks. Finally, to make the ideas and claims more concrete, we demonstrate XAI applied in four specific industrial use cases: commercial vehicles, metro trains, steel plants, and wind farms, spotlighting areas requiring further research.Comment: 51 pages, 9 figure

Learning and Planning of Situated Resource Bounded Agents

This paper presents an investigation of rational agents that have limited computational resources and that can interact with their environments. We analyse how such agents can combine deductive reasoning using domain knowledge and inductive learning from past experiences, while remaining time-aware in a manner appropriate for beings situated in a dynamic universe. In particular, we consider how they can create and reason about partial plans, choose and execute the best ones of them — in such way as to acquire the most knowledge. We also discuss what are the different types of interactions with the world and how they can influence agent’s ability to consciously direct its own learning process

Knowledge Representation for Learning How to Evaluate Partial Plans

In this paper we present some ideas for knowledge representation formalism suitable for rational agents which learn how to choose the best conditional, partial plan in any given situation. In our architecture, the agent uses an incomplete symbolic inference engine, employing Active Logic, to reason about consequences of performing actions — including information-providing ones. It utilises a simple planner to create conditional partial plans, i.e. ones which do not necessarily lead all the way to the ultimate goal. Finally, a learning module — based on ILP mechanisms — provides, from experience, knowledge on how to choose which of those plans ought to be executed. We discuss principles which should guide design of knowledge representations in order to best fit the requirements of learning process. Clearly, simply presenting all of agent’s knowledge to the ILP algorithm is very inefficient. On the other hand, for many particular applications some very effective representations are known. We compare several approaches, analysing the tradeoff between amount of domain specific knowledge provided and the quality of solutions obtained. In the experiments presented we used PROGOL for learning, and one of the conclusions of this paper is that some algorithm better suited for the particular problem of evaluating plans could significantly improve the competitiveness of domain-independent solutions

Learning of agents with limited resources

In our research we investigate rational agent which consciously balances deliberation and acting, and uses learning to augment its reasoning. It creates several partial plans, uses past experience to choose the best one and, by executing it, gains new knowledge about the world. We analyse a possible application of Inductive Logic Programming to learn how to evaluate partial plans in a resource-constrained way. We also discuss how ILP framework can generalise partial plans. Copyrigh

Inductive logic programming algorithm for estimating quality of partial plans

We study agents situated in partially observable environments, who do not have the resources to create conformant plans. Instead, they create conditional plans which are partial, and learn from experience to choose the best of them for execution. Our agent employs an incomplete symbolic deduction system based on Active Logic and Situation Calculus for reasoning about actions and their consequences. An Inductive Logic Programming algorithm generalises observations and deduced knowledge in order to choose the best plan for execution. We show results of using PROGOL learning algorithm to distinguish "bad" plans, and we present three modifications which make the algorithm fit this class of problems better. Specifically, we limit the search space by fixing semantics of conditional branches within plans, we guide the search by specifying relative relevance of portions of knowledge base, and we integrate learning algorithm into the agent architecture by allowing it to directly access the agent's knowledge encoded in Active Logic. We report on experiments which show that those extensions lead to significantly better learning results